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Abstract — Stochastic switching circuits are relay circuits that 
consist of stochastic switches called pswitches. The study of 
stochastic switching circuits has widespread applications in many 
fields of computer science, neuroscience, and biochemistry. In 
this paper, we discuss several properties of stochastic switching 
circuits, including robustness, expressibility, and probability ap- 
proximation. 

First, we study the robustness, namely, the effect caused by 
introducing an error of size e to each pswitch in a stochastic 
circuit. We analyze two constructions and prove that simple 
series-parallel circuits are robust to small error perturbations, 
while general series-parallel circuits are not. Specifically, the total 
error introduced by perturbations of size less than e is bounded 
by a constant multiple of e in a simple series-parallel circuit, 
independent of the size of the circuit. 

Next, we study the expressibility of stochastic switching cir- 
cuits: Given an integer q and a pswitch set S = {|, |, . . . , 3= ^-}, 
can we synthesize any rational probability with denominator q" 
(for arbitrary n) with a simple series-parallel stochastic switching 
circuit? We generalize previous results and prove that when q is 
a multiple of 2 or 3, the answer is yes. We also show that when 
q is a prime number larger than 3, the answer is no. 

Probability approximation is studied for a general case of an 
arbitrary pswitch set S = {si, S2, ■ • • , s s }. In this case, we 
propose an algorithm based on local optimization to approximate 
any desired probability. The analysis reveals that the approxi- 
mation error of a switching circuit decreases exponentially with 
an increasing circuit size. 

Index Terms — Stochastic Switching Circuits, Robustness, Prob- 
ability Synthesis, Probability Approximation. 



I. Introduction 

IN his master's thesis of 1938, Claude Shannon demon- 
strated how Boolean algebra can be used to synthesize and 
simplify relay circuits, establishing the foundation of mod- 
ern digital circuit design lfl2l . Later, deterministic switches 
were replaced with probabilistic switches to make stochastic 
switching circuits, which were studied in [15|. There are a 
few features of stochastic switching circuits that make them 
very similar to neural systems. First, randomness is inherent 
in neural systems and it may play a crucial role in thinking 
and reasoning. Switching (and relaying) technique provides 
us a natural way of manipulating this randomness. Second, 
in a switching system, each switch can be treated as either a 
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memory element or a control element for computing. This 
might enable creating an intelligent system where storage 
and computing are highly integrated. In this paper, we study 
stochastic switching circuits from a basic starting point with 
focusing on probability synthesis. We consider two-terminal 
stochastic switching circuits, where each probabilistic switch, 
or pswitch, is closed with some probability chosen from a 
finite set of rational numbers, called a pswitch set. By selecting 
pswitches with different probabilities and composing them in 
appropriate ways, we can realize a variety of different closure 
probabilities. 

Formally, for a two-terminal stochastic switching circuit C, 
the probabilities of pswitches are taken from a fixed pswitch 
set S, and all these pswitches are open or closed independently. 
We use P(C) to denote the probability that the two terminals 
of C are connected, and call P(C) the closure probability of 
C. Given a pswitch set S, a probability x can be realized if 
and only if there exists a circuit C such that x = P{C). Based 
on the ways of composing pswitches, we have series-parallel 
(sp) circuits and non-series-parallel (non-sp) circuits. An sp 
circuit consists of either a single pswitch or two sp circuits 
connected in series or parallel, see the circuit in Fig. Q~fa) and 
|TJb) as examples. The circuit in Fig. [He) is a non-sp circuit. A 
special type of sp circuits is called simple-series-parallel (ssp) 
circuits. An ssp circuit is either a single pswitch, or is built 
by taking an ssp circuit and adding another pswitch in either 
series or parallel. For example, the circuit in Fig. [TJa) is an 
ssp circuit but the one in Fig. |TJb) is not. 

In this paper, we first study the robustness of different 
stochastic switching circuits in the presence of small error 
perturbations. We assume that the probabilities of individual 
pswitches are taken from a fixed pswitch set with a given error 
allowance of e; that is, the error probabilities of the pswitches 
are bounded by e. We show that ssp circuits are robust to 
small error perturbations, but the error probability of a general 
sp circuit may be amplified by adding additional pswitches. 
These results might help us understand why local errors do 
not accumulate in a natural system, and how to enhance the 
robustness of a system when designing a circuit. 

Next, we study the problem of synthesizing desired proba- 
bilities with stochastic switching circuits. We mainly focus 
on ssp circuits due to their robustness against small error 
perturbations. Two main questions are addressed: (1) Express- 
ibility: Given the pswitch set S = {|, . . . , ^— }, where 
q is an integer, what kind of probabilities can be realized 
using stochastic switching circuits? And how many pswitches 
are sufficient to realize them? (2) Approximation: Given an 
arbitrary pswitch set S, how can we construct a stochastic 
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(a) ssp circuit. P(C) = §. 
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(b) sp circuit, non-ssp. 
P{C) = 
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(c) non-sp circuit. P(C) = ^. 
Fig. 1. Examples of ssp, sp, and non-sp circuits. 



switching circuit using as a few as possible pswitches, to get 
a good approximation of the desired probabilities? 

The study of probability synthesis based on stochastic 
switching circuits has widespread applications. Recently, peo- 
ple found that DNA molecules can be constructed that closely 
approximate the dynamic behavior of arbitrary systems of 
coupled chemical reactions 0131 . which leads to the field 
of molecular computing (2). In such systems, the quantities 
of molecules involved in a reaction are often surprisingly 
small, and the exact sequence of reactions is determined by 
chance [4|. Stochastic switching circuits provide a simple and 
powerful tool to manipulate stochasticity in molecular sys- 
tems. Comparing with combinational logic circuits, stochastic 
switching circuits are easier to implement using molecular re- 
actions. Another type of applications is probabilistic electrical 
systems without sophisticated computing components. In such 
systems, stochastic switching circuits have many advantages 
in generating desired probabilities, including its constructive 
simplicity, robustness, and low cost. 

The remainder of this paper is organized as follows: Section 
[H] describes related work and introduces some existing results 
on stochastic switching circuits. In Section [Till we analyze the 
robustness of different kinds of stochastic switching circuits. 
Then we discuss the expressibility of stochastic switching 
circuits in Section[IV]and probability approximation in Section 
[Vl followed by the conclusion in Section [VI] 

II. Related Works and Preliminaries 

There are a number of studies related to the problem of 
generating desired distributions from the algorithmic perspec- 
tive. This problem dates back to von Neumann fl4l . who 
considered of simulating an unbiased coin using a biased coin 
with unknown probability. Later, Elias [ 3 1 improved this algo- 
rithm such that the expected number of unbiased random bits 
generated per coin toss is asymptotically equal to the entropy 
of the biased coin. On the other hand, people have considered 
the case that the probability distribution of the tossed coin is 



known. Knuth and Yao [9| have given a procedure to generate 
an arbitrary probability using an unbiased coin. Han and Hoshi 
J2l have demonstrated how to generate an arbitrary probability 
using a general M-sided biased coin. All these works aim to 
efficiently convert one distribution to another. However, they 
require computing models and may not be applicable for some 
simple or distributed electrical/molecular systems. 

There are a number of studies focusing on synthesizing a 
simple physical device to generate desired probabilities. Gill 
discussed the problem of generating rational proba- 
bilities using a sequential state machine. Motivated by neural 
computation, Jeavons et al. provided an algorithm to generate 
binary sequences with probability from a set of stochastic 
binary sequences with probabilities in {i, ^— } 10. 

Their method can be implemented using the concept of linear 
feedback shift registers. Recently, inspired by PCMOS tech- 
nology (JJ, Qian et al. considered the synthesis of decimal 
probabilities using combinational logic ifTTTl . They have con- 
sidered three different scenarios, depending on whether the 
given probabilities can be duplicated, and whether there is 
freedom to choose the probabilities. In contact to the forego- 
ing contributions, we consider the properties and probability 
synthesis of stochastic switching circuits. Our approach is or- 
thogonal and complementary to that of Qian and Riedel, which 
is based on combinational logic. Generally, each switching 
circuit can be equivalently expressed by a combinational logic 
circuit. All the constructive methods of stochastic switching 
circuits in this paper can be directly applied to probabilistic 
combinational logic circuits. 

In the rest of this section, we introduce the original work 
that started the study on stochastic switching circuits (Wilhelm 
and Bruck |[T5l ). Similar to resistor circuits |[Tol . connecting 
one terminal of a switching circuit C\ (where P{C\) — p\) to 
one terminal of a circuit C2 (where P(C2) — P2) places them 
in series. The resulting circuit is closed if and only if both 
of C\ and C2 are closed, so the probability of the resulting 
circuit is 

Pscrics = Pl ■ P2- 

Connecting both terminals of C\ and C2 together places the 
circuits in parallel. The resulting circuit is closed if and only 
if either C\ or C2 is closed, so the probability of the resulting 
circuit is 

^parallel = 1 - (1 ~ Pl)(l ~ Pa) = Pi + P2 ~ P\P2- 

Based on these rules, we can calculate the probability of any 
given ssp or sp circuit. For example, the probability of the 
circuit in Fig. [Tfa) is 



P(a) 



and the probability of the circuit in Fig. [TJb) is 
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Let us consider the non-sp circuit in Fig. [Tic). In this circuit, 
we call the pswitch in the middle a 'bridge'. If the bridge is 
closed, the circuit has a closure probability of j^. If the bridge 
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is open, the circuit has a closure probability of jg. Since the 
bridge is closed with probability |, the overall probability of 
the circuit is 

19 17 1 
P[c) ~2'l6 + 2'l6~2' 
An important and interesting question is that if S is uniform, 
i.e., S = {-, for some q, what kind of probabili- 

ties can be realized using stochastic switching circuits? In 1 15 1, 
Wilhelm and Bruck proposed an optimal algorithm (called B- 
Algorithm) to realize all rational probabilities of the form 
with < a < 2™, using an ssp circuit when S = {|}. In their 
algorithm, at most n pswitches are used, which is optimal. 
They also proved that given the ps witch set S = {|, |}, all 
rational probabilities ^ with < a < 3" can be realized by 
an ssp circuit with at most n pswitches; given the pswitch set 
S = {i, §, §}, all rational probabilities -jfc with < a < 4" 
can be realized by an ssp circuit with at most 2n — 1 pswitches. 





(a) Initial circuit. P ■■ 



Fig. 2. A circuit and its dual. 



(b) The dual. P - 



Wilhelm and Bruck also demonstrated the concept of duality 
in sp circuits. The dual of a single pswitch of probability p 
appearing in series is the corresponding pswitch of probability 
1 — p appearing in parallel. Similarly, the dual of a pswitch of 
probability p appearing in parallel is a pswitch of probability 
1 — p appearing in series. For example, in Fig. [2] the circuit 
in (b) is the dual of the circuit in (a), and vice versa. It can 
be proved that dual circuits satisfy the following relation: 

Theorem 1 (Duality Theorem IflSI ). For a stochastic series- 
parallel circuit C and its dual C, we have 

p(c) + p(a) = i, 

where P(C) is the probability of circuit C and P{C) is the 
probability of circuit C. 

III. Robustness 

In this section, we analyze the robustness of different kinds 
of stochastic switching circuits, where the probabilities of 
individual pswitches are taken from a fixed pswitch set, but 
given an error allowance of e; i.e., the error probabilities 
of the pswitches are bounded by e. For a stochastic circuit 
with multiple pswitches, the error probability of the circuit 
is the absolute difference between the probability that the 
circuit is closed when error probabilities of pswitches are 
included, and the probability that the circuit is closed when 
error probabilities are omitted. We show that ssp circuits are 
robust to small error perturbations, but the error probability 
of a general sp circuit may be amplified with additional 
pswitches. 



A. Robustness of ssp Circuits 

Here, we analyze the susceptibility of ssp circuits to small 
error perturbations in individual pswitches. Based on our 
assumption, instead of assigning a pswitch a probability of 
p, the pswitch may be assigned a probability between p — e 
and p + e, where e is a fixed error allowance. 

Theorem 2 (Robustness of ssp circuits). Given a pswitch set 
S, if the error probability of each pswitch is bounded by e, 
then the total error probability of an ssp circuit is bounded by 

c 



min(min(S'), 1 — max(S')) 

Proof: We induct on the number of pswitches. If we 
have just one pswitch, the result is trivial. Suppose the 
result holds for n pswitches, and note that for an ssp circuit 
with n + 1 pswitches, the last pswitch will either be added 
in series or in parallel with the first n pswitches. By the 
induction hypothesis, the circuit constructed from the first n 
pswitches has probability p + t\ of being closed, where e\ 
is the error probability introduced by the first n pswitches 



and | e 1 1 < 



The (n + l)st pswitch has 



min(min(S') , 1 — max(S)) ' 

probability t + e 2 of being closed, where t E S and | €2 1 < £■ 



p + s, 



t + £ n 



p + e x 



(a) The last pswitch is added in 
series. 

Fig. 3. Robustness of ssp circuits. 



t + £ 2 

(b) The last pswitch is added in 
parallel. 



If the (n + l)st pswitch is added in series, see Fig. [3| a X 
then the new circuit (with errors) has probability 

(p + ei)(t + e 2 ) = tp + € 2 {p + ei) + tei 

of being closed. Without considering the error probability of 
each pswitch, the probability of the new circuit is tp. Hence, 
the overall error probability of the circuit is e\ = e 2 {p + ei) + 
tt\. By the triangle inequality and the induction hypothesis, 



ei 



< 
< 

< 
< 



|c 2 ||(p + ei)|+*|ei| < M+*|ei 

1 I > 



min(min(,5'), 1 — max(5)) 

min(min(S'), 1 — max(S')) + max(S') 

min(min(5), 1 — max(S')) 
e 



min(min(5), 1 — max(S')) 

completing the induction. 

Similarly, if the (n + l)st pswitch is added in parallel, see 
Fig. Ob), then the new circuit (with errors) has probability 

(p + ei) + (t + e 2 ) - (p + ei)(t + e 2 ) 
= (p + t-tp) + e 1 (l-t) + e 2 (l-p-e 1 ) 

of being closed. Without considering the error probability of 
each pswitch, the probability that the circuit is closed is p + 
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t — tp. Hence, the overall error probability of the circuit with 
n + 1 pswitches is 62 = ei(l — t) + £2(1 — P — Again 
using the induction hypothesis and the triangle inequality, we 
have 



|e 2 | 

< | ea ||(i_ p _ ei )| + (i_t)| ei | 

< | ca |+(i-t)| ei | 
1 - t 

min{min(5), 1 — max(S')) 
^ min{min(S f ), 1 — max(iS)) + 1 — min(S') 



< 



1 e 



< 



min{mm(S'), 1 — max(S')) 
e 



min{min(5 f ), 1 — max(iS)) 

This completes the proof. ■ 

The theorem above implies that ssp circuits are robust to 
small error perturbations: no matter how big the circuit is, the 
error probability of an ssp circuit will be well bounded by a 
constant times e. Let us consider a case that S = {^}. In this 
case, the overall error probability of any ssp circuit is bounded 
by 2e if each pswitch is given an error allowance of e. 

B. Robustness of sp Circuits 

We have proved that for a given pswitch set S, the overall 
error probability of an ssp circuit is well bounded. We want to 
know whether this property holds for all sp circuits. Unfortu- 
nately, we show that as the number of pswitches increases, the 
overall error probability of an sp circuit may also increase. In 
this subsection, we will give the upper bound and lower bound 
for the error probabilities of sp circuits. 

Theorem 3 (Lower bound for sp circuits). Given a pswitch set 
S, if the error probability of each pswitch is e (where e — > 0), 
then there exists an sp circuit of size n with overall error 
probability 0(logn)e. 

Proof: Suppose p £ S, and without loss of generality, 
assume e > 0. We construct an sp circuit as shown in Fig. |4] 
by connecting a + 1 strings of pswitches in parallel. Among 
these strings, we have a strings of b pswitches and one string 
of n—ab pswitches, and all pswitches have probability p. Now, 
we let a and b satisfy the following relation: 



— 1, a = 



(~) b 
. P . 



Without considering pswitch errors, the probability of the 
circuit is 

Pi = l-(l-p b ) a (l-p n - ab ). 

Suppose we introduce an error of e to each pswitch, such 
that the probability of each pswitch is p + e (assume e > 0). 
Then the probability of the circuit is 

p 2 (e) = 1 - (1 - (p + - (p + e) n - ab ), 

where P2(0) = pi- 



b pswitches 
» 



-0—0- 



-0Mgh 



-0—0 



-0-0- 



-0—0 — 



-0-0- 



-0—0 



a + 1 strings 



1 

n-ab pswitches 

Fig. 4. The construction of an sp circuit. 

Assuming n is large enough, we have the following error 
probability for the circuit: 

ei 

= Pa(e)-pi 
^ P'^Y 

~ -[{l-{p + e) b Y{l-{p + e) n - ab )]'e 



-a(p+e) f 



(l-(p + eT- ab )]'e 



e - a{p+e)b ab(p + ef-^l - (p+ e) n - ab )e 



abp b - l {l - p n - ab ) + e- ap (n - ab)p n - a "- 1 ]t 

~ [^-(l - p n - ab ) + e-\n - ab)p n - ab - x ]e. 
P 

So when n is large enough, we have 

e^^—^be < | ei I < e~ l -be. 
P P 

Since b[(±) b \ <n< b{l(±) b \ + 1) for large n, we have 
log n log log n 1°§ l°g p log n 



log- 



log - log : 



Finally, we have |ei| ~ 0(logn)e, completing the proof. ■ 

In the following theorem, we will give the upper bound for 
the error probabilities of sp circuits. 

Theorem 4 (Upper bound for sp circuits). Given an sp circuit 
with n pswitches taken from a finite pswitch set S, if each 
pswitch has error probability bounded by e, then the total 
error probability of the circuit is bounded by c^/ne, where 



c = maxtgs 



is a constant. 



Proof: Assume x is a pswitch in a stochastic circuit C, 
and the actual probability of x is t x + e x , where e x is the error 
part such that \e x \ < e. Let P(C\x = 1) denote the probability 
of circuit C when x is closed, and let P(C\x = 0) denote the 
probability of C when x is open. 

Without considering the error probability of x, the proba- 
bility of circuit C can be written as 

P X {C) = t x P(C\x = !) + (!- t x )P(C\x = 0). 
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Considering the error part of x, we have 

P(C) = (t x + e x )P{C\x = 1) + (1 - t x - e x )P{C\x = 0). 

In order to prove the theorem, we define a term called the 
error contribution. In a circuit C, the error contribution of 
pswitch x is defined as 

e x (C) = \P(C) - P X (C)\ = e x \P(C\x = 1) - P(C\x = 0)| 

< e(P{C\x = 1) - P(C\x = 0)). 

In the rest of the proof, we have two steps. 
(1) In the first step, we show that given an sp circuit 
with size n, there exists at least o ne psw itch such that its 

error contribution is bounded by c '^ ( - 1 p - > _ e where P is the 

probability of the sp circuit and c = max te s —j=^ . 

V *(!—*) 

We induct on the number of pswitches. If the circuit has 
only one pswitch, the result is trivial. Suppose the result holds 
for k pswitches for all k < n. We need to prove that the result 
holds for any sp circuit C with n pswitches. 

Suppose circuit C is constructed by connecting two sp 
circuits C\ and C 2 in series, where C\ has n\ pswitches and 
probability Pi, and C 2 has n 2 pswitches and probability P2. 
Note that n\ + n 2 = n and n\ < n,ri2 < n. 

By the induction hypothesis, circuit C\ contains a pswitch 
x\ with error contribution 



which can be simplified as 



e Xl {Ci) < — — e. 



In circuit C, the error contribution of pswitch X\ is 

e Xl (C) = |P(C) - P X1 (C)\ = P 2 \P{C l ) - P XX {C X )\ 
= P 2 e Xl (C 1 ). 

Similarly, in the circuit C2, there exists a pswitch x 2 such 
that the error contribution of x 2 is 



e x 2 \ ( ^2) S . e, 



n 2 

and the error contribution of x 2 to circuit C is 

e X2 (C) =P iex2 (C 2 ). 

Since the circuit C is constructed by connecting circuits G\ 
and C 2 in series, the probability of circuit C is P — P\P 2 . 
Thus, we only need to prove that either e Xl (C) or e X2 (C) is 
bounded by 

Cy/0. - PiP 2 )PiP 2 

/ T e ' 

y/rii + n 2 

This can be proved by contradiction as follows. 

Assume both e xl (C) and e X2 (C) are larger than 

cJ(l-PlP 2 )PlP2 ™ , 

— - — . e. Then we have 

VH+12 



p cV(l-Pi)Pi Cv /(1 - PiF 2 )PiP 2 
"2 . > 



and 



D Cv /(1 - P 2 )P 2 cV(l - PlP 2 )PlP 2 

r i 7= > / ■ 

y/ni + n 2 



"1 



< 



and 



(l-Pi)P 2 
ni +n 2 " 1 — P1P2 

(1 - P 2 )Pi 



"2 



< 



ni+n 2 \-P x P 2 
Adding the two inequalities yields 

Pi + P 2 - 1 - P1P2 = -(1 - Pi)(l - Pi) > 0, 

which is a contradiction. So we conclu de that at le ast one 
of e Xl (C) and e X2 (C) is bounded by c ^ '- Pl P2 e when 

C is constructed by connecting two sp circuits in series. If 
the circuit C is constructed by connecting two sp circuits 
in parallel, using a similar argument, we can get the same 
conclusion. 

Finally, we get that given an sp circuit with size n, there 
exists at least one pswitch such that its error contribution is 
bounded by £VAi^ZlZ e 

(2) In the second step, we prove the theorem based on the 
result above. 

We again induct on the number of pswitches. If we have 
less than three pswitches, the result is trivial. Suppose the 
result holds for any sp circuit with n > 2 pswitches; we want 
to prove that the result also holds for any circuit with n + 1 
pswitches. 

Based on the result in the first step, we know that given an 
sp circuit C with n + 1 pswitches, there exists a pswitch x 
with error contribution bounded by - , c , , e. 

J 2 \/n+l 

By keeping pswitch x closed, we obtain an sp circuit 
D\ with at most n pswitches. Please see Fig. |3a)(b) as an 
example. Without considering pswitch errors, D\ is closed 
with probability p±; considering all pswitch errors, D\ is 
closed with probability q\. According to our assumption, we 
have 

ei = \qi -V\\ < cVne. 



A 

I — ®- 



B 



C 



D 




(a) Circuit C. 



(b) Di, A closed. 



c 



D 



(c) D2, A open. 

Fig. 5. An illustration of keeping a pswitch A closed or open in an sp circuit 
C. 
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By keeping pswitch x open, we obtain an sp circuit D2 
with at most n pswitches. Please see Fig. |5ja)(c) as an 
example. Without considering pswitch errors, D2 is closed 
with probability P2, considering all pswitch errors, D2 is 
closed with probability (72- According to our assumption, we 
have 

e2 = 1 32 -P2I < cy/ne. 

For the initial sp circuit C with n + 1 pswitches, without 
considering pswitch errors, the overall probability of the circuit 
is given by 

txPl + (1 - t x )p 2 , 

where t x is the probability of pswitch x. 

Considering all pswitch errors, the overall probability of the 
circuit is 

{t x + (-x)qi + (1 - t x - e x )q 2 . 

We know that the error contribution of pswitch x to the 
circuit C is 

e x (C) = ex\q-2 - qi\ < 



Then by the triangle inequality, we can get the error 
probability of the circuit C: 



= \(t x + e x )qi + (l-t x - e x )q2 - (t x Pi + (1 - t x )p 2 )\ 
< t x \qi -pi\ + (1 - t x )\q 2 -F2I + £ x \q2 - qi\ 



< 



cy/n{n + 1) + f 
Vn+1 
+ 5 

cyn + le. 



This finishes the induction. 



C. Robustness of Non-sp Circuits 

Here we extend our discussion to the case of general 
stochastic switching circuits. We have the following theorem, 
which clearly holds for sp and ssp circuits: 

Theorem 5 (Upper bound for general circuits). Given a gen- 
eral stochastic switching circuit with n pswitches taken from 
a finite pswitch set S, if each pswitch has error probability 
bounded by e, then the total probability of the circuit is 
bounded by ne. 

Proof: We first index all the pswitches in the circuit C 
as x\, X2, • • • , x n , see Fig. [6] as an example. 

Let ti+ei be the probability that Xi is closed, where ej is the 
error part such that |e{| < e. Let denote the probability 
that C is closed when we only take into account the errors of 
xi,x 2 , ...,Xk, i.e., 

p( fc ) = P(ti+ei,...,t k + ek,t k+ i,...,t n ), 

where P{a\, 0,2, ■ ■ ■ , a n ) indicates the probability of C if x^ 
is closed with probability dj for all 1 < i < n. 



®*3 - 



Fig. 6. An example of a general stochastic switching circuit. 

The overall error probability of the circuit C can then be 
written as 

e = P {n) - P(0) 

= (p(») _ p(n-l)\ _|_ fp(n-l) _ p(n-2)\ _j_ . . . 
+ (p(l) _ p(0)). 

Now, we prove that |P (fc) - P^ 1 ) | < e for all 1 < k < n 

ip(fe) _ p (fc-i)| 

= \P(ti +ei,...,t k + e k ,t k+ i,. . .,t n ) 
—P(ti + ei, . . . , tfe_i + c k -i,t k , . . . , i n )| 

= |(*fc + £fc)P(*l + ei, . 1, tfc+i, ■ • ■ ,t n ) 

+ (1 - t k ~ e k )P(h +e u ..., 0, tk+i, ■ • • , t n ) 

—t k P(tx + €1, . . . , t k _i + £fc_i, 1, . . . , t n ) 

-(1 -t k )P{tl +€!,..., t k -! +£fe-l,0, ...,t n )\ 

= \e k [P(ti + ei, . . . , 1, tfc+i, • ..,*«) 

-P(tl + e 1 ,...,0,tfc+l,..-,*n)]| 

< e. 
Therefore, we have 

n 

e <J2\P (k) -P ik ~ 1] \ <ne, 

k=l 

as we wanted. ■ 

Note that in most of cases, the actual error probability of a 
circuit is much smaller than ne when n is large. However, ne 
is still achievable in the following case: by placing n pswitches 
with probability p — e in series, where e —> 00, we can get a 
circuit whose probability is 

(p - e) n pa p n - np n ~ l e. 

Without considering the errors, the probability of the circuit 
is p n , so the overall error is n-p n ~ 1 e. Choosing p sufficiently 
close to 1, we can make the error probability of the circuit 
arbitrarily close to ne. 

IV. EXPRESSIBILITY 

In the previous section, we showed that ssp circuits are 
robust against noise. This property is important in natural 
systems and useful in engineering system design, because 
the local error of a system should not be amplified. In this 
section, we consider another property of stochastic switching 
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TABLE I 

The expressibility of stochastic switching circuits 





Can all be realized? 


upper bound of circuit size 


q is even 


yes, ssp circuit 


|"log 2 ? l(n-l) + l 


g is an odd multiple of 3 


yes, ssp circuit 


flogg q](n -1) + 1 


q is a prime number larger than 3 


no, not by sp circuits 




other values of q 


open problem 





circuits, called expressibility. Namely, given a pswitch set 
S = {|, -, . . . , } for some integer q, the questions we 
ask are: What kinds of probabilities can be realized using 
stochastic switching circuits (or only ssp circuits)? How many 
pswitches are sufficient? Wilhelm and Bruck IT5l proved that 
if q — 2 or q = 3, all rational with < a < q n , can be 
realized by an ssp circuit with at most n pswitches, which is 
optimal. They also showed that if q = 4, all rational with 
< a < q n , can be realized using at most 2n — 1 pswitches. 
In this section we generalize these results: 

1) If q is an even number, all rational -2-, with < 
a < q n , can be realized by an ssp circuit with at most 
[log 2 q](n — 1) + 1 pswitches (Theorem [7]). 

2) If q is odd and a multiple of 3, all rational with 
< a < q n , can be realized by an ssp circuit with at 
most [log 3 q~\ (n — 1) + 1 pswitches (Theorem[8]l. 

3) However, if q is a prime number greater than 3, there 
exists at least one rational with < a < q n , that 
cannot be realized using an sp circuit (Theorem fTTb. 

Table U summarizes these results. We see that when q = 2,3, 
or 4, our results agree with the results in fl5l . 




(a) Step 1: We assume that the de- 
sired probability is pi. 





Pi 





(c) Step 3: Insert X2 in series as 
the last pswitch. Now we try to 
realize p:j such that P3X2 = P2- 





Pi 











(b) Step 2: Insert x± in parallel as 
the last pswitch. Now we try to re- 
alize P2 such that P2+X1 —p2%l = 
Pi- 



(d) Last step: Replace p$ with a 
single pswitch x$. 



Fig. 7. An example of the backward algorithm. 



A. Backward Algorithms 

As mentioned in fl31 . switching circuits may be synthesized 
using forward algorithms, where circuits are built by adding 
pswitches sequentially, or backward algorithms, where circuits 
are built starting from the "outermost" pswitch. 

Fig- 13 gives a simple demonstration of a backward al- 
gorithm. Assume that the desired probability is p\ and we 
plan to insert three pswitches, namely xi,X2,x% in backward 
direction. Here, for simplicity, we use xi,X2,x^ to denote 
the closure probabilities of the pswitches, rather than their 
states (1 or 0). If x\ < p\, then x\ has to be inserted in 
parallel. If x\ > pi, then x\ has to be inserted in series. 
After the insertion, we can try to realize the inner box with 
probability P2 such that P2 + x\ — P2X1 = p%. This process 
is continued recursively until for some m, p m can be realized 
with a single pswitch. Generally, in backward algorithms, we 
use Xk to denote the fcth pswitch inserted in the backward 
direction, and use pk to denote the probability that we want 
to realize with pswitches Xk, Xk+i, Xk+2, ■ ■ ■ 

Backward algorithms have significant advantages over for- 
ward algorithms for probability synthesis. In a forward algo- 
rithm, if we want to add one pswitch, we have 2|5| choices, 
since each pswitch may be added in either series or parallel. 
But in a backward algorithm, if we want to insert one pswitch, 
we have only |5| choices. That is because the insertion (series 
or parallel) of a pswitch Xk simply depends on the comparison 



of Xk and pk - Therefore, backward algorithms can significantly 
reduce the search space, hence are more efficient than forward 
algorithms. In this paper, most of the circuit constructions are 
based on backward algorithms. 



q-l 



} and q is 



B. Multiples of 2 or 3 

We consider the case that S = { ^ , | , 
a multiple of 2 or 3. We show that based on a backward 
algorithm, all rational with < a < q n , can be realized 
using a bounded number of pswitches. Before describing the 
details, we introduce a characteristic function called d for a 
given probability ^7, that is 



gcd(fe, q™- 1 )' 



Note that the function d is well defined, i.e., the value of d 
is unchanged when both b and q w are multiplied by the same 
constant. From the definition of the characteristic function d, 
we see that for any rational with < a < q n , d is a 
positive integer. In each iteration of the algorithm, we hope to 
reduce d(pk) such that it can reach 1 after a certain number of 
iterations. If d = 1, this means the desired probability can be 
realized using a single pswitch and the construction is done. 
During this process, we keep each successive probability pk 
in the form of ^5-, since only this kind of probabilities can 
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(a) Step 1 with d(pi) = 10. 



5 

10 



10 



(c) Step 3 with d(p 3 ) = 2. 





275 











10 

(b) Step 2 with d(p 2 ) = 4. 



5 

To 

H8h 



Hg>7 



10 



(d) Step 4 with d(p 4 ) = 1. 



h(x,p k ) 




(a) e((p fc ) is even and p k < \. 



Hx,p t ) 




(c) d(p k ) is odd and p k < §. 



Kx,p k ) 



— 1 

(b) d(p fe ) is even and p fc > 1. 



-0- 



9 -2 s 



(d) c((p fe ) is odd and p fc > 1. 



Fig. 8. The procedure to realize j^- for a given pswitch set S 
1 10 10 10 J 



Fig. 9. When g is even, the way to add a pswitch x G S such that 
d(h(x,p k )) < d(p k ). 



be realized with the pswitch set S. Now, we describe the 
algorithm as follows. 



Algorithm 1 (Backward algorithm to realize pi = -^k with 

1 



{I 2 



< a < g" and pswitch set 5* 

1) Se? fc = 1, starting with an empty circuit. 

2) Let 

Hi. 

h{x kl Pk 



-})• 



£f if x k > p k (series), 

k 

Pk ~ Xk if Xk < p k (parallel). 



1 — Xfc 



We find the optimal x k € S 1 fnaf minimizes d(p, 
Pk+i = h(x k ,p k ). Ifpk+i = 



k+l 



with 



d(p. 



then 
q w 



gcd(6, g 1 "- 1 )' 

3) Insert pswitch Xk to the circuit. If x k > p k , the pswitch 
is inserted in series; otherwise, it is inserted in parallel. 
Then we set p k+ i = h(x k ,Pk)- 

4) Let k = k + 1. 

5) Repeat steps until p k can be realized using a single 
pswitch. Then insert p k into the circuit. 

In Algorithm [U the characteristic function d(p k ) strictly 
decreases as k increases, until it reaches 1. Finally, p k can 
be replaced by a single pswitch and the construction is done. 
Fig.[8]gives an example of a circuit realized by this algorithm. 
At the beginning, we have p\ = ji?, with d(pi) = 10. 
Then we add the "best" pswitch to minimize d(p2), where the 



optimal pswitch is jq. Since jjj < we insert the pswitch 



6 ^ 71 

in parallel, making d{p2) = 4. Repeating this process, we have 
71 275 55 1 

Pl = W P2 = W P3 ^W' P4= 10' 

with corresponding characteristic functions 

d(p x ) = 10, d( P2 ) = 4, d{p 3 ) = 2, d(p 4 ) = 1. 

In the following theorem, we show that if q is a multiple 
of 2 or 3, then Algorithm [TJ realizes any rational -jjj- with 
< a < q n . 



Theorem 6. Given a pswitch set S — { | , | , . . . , }, if q 
is a multiple of 2 or 3, then Algorithm\l\realizes any rational 
w/f/z < a < g™, wsi'ng an ssp circuit with a finite number 
of pswitches. 

Proof: The characteristic function d{p\) of the initial 
probability p\ is bounded by We only need to prove that 

there exists an integer m such that d(p m ) = 1, i.e., p m can be 
realized by a single pswitch. Hence the desired probability p\ 
can be realized by an ssp circuit with m pswitches. It is enough 
to show that the characteristic function d(p k ) decreases as k 
increases. 

First, we consider the case where q is even. We will 
show that for any p k = there exists x £ S such that 
d(h(x,p k )) < d(pk). See Fig. |£l depending on the values 
of pk and d(pk), we have four different cases of inserting a 
pswitch x such that d(h(x,p k )) < d(p k ). 

1) If d(pk) is even and pk < \, let x 
pswitch in series. 

2) If d(p k ) is even and p k > 
pswitch in parallel. 

3) 



| and insert the 



let x = h and insert the 



If d(p k ) is odd and p k < |, 



and insert the pswitch in series. 



let x = y with s = [\og 2 q\ 



- g- 2 

2' — g 

[log 2 and insert the pswitch in parallel. 



4) If d(p k ) is odd and p k > i, let x 



with 



By checking all the cases to insert a pswitch, it is straight- 
forward to see that when d(p k ) is even, d(h(x,p k )) < \d{p k ), 
and when d(p k ) is odd, 



d(h(x,p k )) < 



2 s d( Pk ) 



< d(p k ). 



icd(q,2*d(pk)) 
Since Xk is optimal in each step of Algorithm Q] we have 

d{p k +i) = d(h(x k ,Pk)) < d(h(x,p k )) < d(p k ). 

Finally, we can conclude that when q is even, there exists 
an integer m such that d(p m ) = 1. Consequently, pi can be 
realized with at most m pswitches. 
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optimal size=5 



optimal size=4 

.0 9 Q = 

optimal size=3 



Fig. 10. For each q, the average number of ps witches used in Algorithm \T\ 
to realize the rational probabilities when their optimal size is n. 



Similarly, when q is odd and a multiple of 3, if p k = 
we can always insert a pswitch x € S such that d(h(x,pk)) < 
d(pk), as follows: 

1) If d(pk) mod 3 = and p k < \, let x = |, and insert 
the pswitch in series. 

2) If d(pk) mod 3 = and | < p k < | with even b, let 
x = |, and insert the pswitch in series. 

3) If d(pfc) mod 3 = and | < p fe < | with odd b, let 
x = |, and insert the pswitch in parallel. 

4) If d(pk) mod 3 = and p k > |, let x = |, and insert 
the pswitch in parallel. 

5) If d(pk) mod 3 7^ and p fe < |, let a; = — with 
s = [\og 3 q\ , and insert the pswitch in series. 

6) If d(pk) mod 3^0 and | < p k < | with even 6, let 
a; = with s = |l°g3 9 J > an d insert the pswitch in 
series. 

7) If d(p k ) mod 3 7^ and | < p fe < | with odd b, let 
a; = 3 with s = [log 3 q\ , and insert the pswitch in 
parallel. 

8) If d{pk) mod 3^0 and p k > §, let a; = with 
s = Ll°g3 l\ > an d insert the pswitch in parallel. 

Finally, we can conclude that p\ can be realized with a finite 
number of pswitches when q is odd and a multiple of 3. ■ 



For each value q € {2,3,4,6,8,9,10}, we enumerate all 
rational numbers with optimal size n 6 (3,4,5). Here, we 
say that a desired probability is realized with optimal size if 
it cannot be realized with fewer pswitches. As a comparison, 
we use Algorithm QJ to realize these rational numbers again. 
Fig. [10] presents the average number of pswitches required 
using Algorithm QJ when the optimal size is n. It is shown 
that when q is a multiple of 2 or 3, Algorithm [TJ can construct 
circuits with almost optimal size. 

The next theorem gives an upper bound for the size of the 
circuits when q is even. 



any rational with < a < q n can be realized by an ssp 
circuit, using at most [log 2 q~\ (n — 1) + 1 pswitches. 

Proof: In order to achieve this upper bound, we use 
a modified version of Algorithm [TJ Instead of inserting the 
optimal pswitch x k , we insert the pswitch x described in Fig. [9] 
as the fcth pswitch. The resulting characteristic function has the 
following properties: 

(1) d(pk) decreases as k increases, and when d(p m ) = 1 
for some m, the procedure stops. 

(2) If d(pk) is even, then d(pk+i) is a factor of d . 

(3) If d(pk) is odd, then d(p k +i) is a factor of g C( j^ vd(p k )) ■ 
We define 

N = min{fc|A: G (1, 2, 3, . . .), d(p k ) = 1}, 

then N is the number of required pswitches. We only need to 
prove that N < \\og 2 <?] (n — 1) + 1. Since q is even, we can 
write q = 2 C or q = 2 c t, where t > 1 is odd. 

Let us first consider the case of q = 2 C . At the beginning, 
d(pi) is a factor of q ,n_1 , so according to property (2), we can 
get 

N < c(n — 1) + 1 = [log 2 q] (n - 1) + 1. 

In the case of q — 2 c t, let us define a set M as 

M = {k\k > 0,d{ Pk ) is odd}, 

and let Mi be the ith smallest element in M. According to 
properties (2) and (3) and the fact that d(pi) is a factor of 
q™ -1 , we see that d(pM t ) is a factor of q n ~ % . Therefore, there 
exits a minimal k, with k < n, such that d(pM k ) = 1- Then 
= M k . 

Based on properties (2) and (3), we also see that 



and 



Therefore, 



N < 



Mi < c(n - 1) + 1, 



M l+1 - Mi <s-c. 



i=l 

flog 2 q] (n 



This completes the proof. 



Mi) + Mi < s(n ■ 
1) + 1. 



Using the similar methods, we can prove the following 
theorems as well when q is a multiple of 3 or 6. Note that 
Theorem [7] also applies to the case that q is a multiple of 6, 
but Theorem [9] provides a tighter upper bound. 

Theorem 8 (Upper bound of circuit size when q is odd and 
a multiple of 3). Given a pswitch set S = { ~ , ~ , . . . , }, 
if q is odd and a multiple of 3, then any rational with 
< a < q n can be realized using an ssp circuit with at most 
[log 3 q\ (n — 1) + 1 pswitches. 



Theorem 7 (Upper bound of circuit size when q is even). Theorem 9 (Upper bound of circuit size when q is a multiple 



Suppose q is even. Given a pswitch set S = {|, |, . . . , 



of 6). Given a pswitch set S — {jj'gj 



9-1 



}, ifq 
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multiple of 6, all rational with < a < q n can be realized 
by an ssp circuit with at most N pswitches, where 

f (2s)(n-l) + l (if6 s =q), 

I (2a + l)(n-l) + l (iff <6 S <q), 

iVS 1 ( 2s + 2)(n-l) + l (if | < 6 s < |), 

[ (2s + 3)(n-l) + l (;/ § < 6 s < §). 

C. Prime Number Larger Than 3 

We proved that if q is a multiple of 2 or 3, all rational 
can be realized with a finite number of pswitches. We want to 
know whether this result also holds if q is an arbitrary number 
greater than 2. Unfortunately, the answer is negative. 

Lemma 10. Suppose q is a prime number. Given a pswitch 
set S — { , ^— }, if a rational cannot be realized 
fry an sp circuit with n pswitches, then it cannot be realized 
using an sp circuit with any number of pswitches. 

Proof: Assume there exits a rational which cannot be 
realized by an sp circuit with n pswitches, but can be realized 
with at least I > n pswitches. Further, suppose that this I is 
minimal for all rationals with denominator q k . Under these 
assumptions, we will prove that there exists a rational -^7- 
which cannot be realized with n' pswitches but can be realized 
with I' pswitches such that I' < I. This conclusion contradicts 
the assumption that I is minimal. 

According to the definition of sp circuits, we know that 
can be realized by connecting two sp circuits C\ and C 2 in 
series or in parallel. Assume C\ consists of l\ pswitches and 
is closed with probability and C 2 consists of I2 pswitches 
and is closed with probability where l\+l 2 = I- 

If C\ and C 2 are connected in series, we can get 

bi b 2 _ a 
qh q h qn " 

Therefore, 6162 = aq l ~ n , where 6162 is a multiple of q. 
Since q is a prime number, either b\ or 62 is a multiple of q. 
Without loss of generality, assume b\ is a multiple of q, and we 
write bi = cq. Consider the probability ^rf=i > which can be 
realized with Ci, using l\ pswitches. Assume that the same 
probability can also be realized with another sp circuit C3, 
using l\ — 1 pswitches. By connecting C3 and C 2 in series, we 
can realize with l\ — 1 +Z2 = I — 1 pswitches, contradicting 
the assumption that cannot be realized with less than I 
pswitches. Therefore, we see that cannot be realized 

with 1% — 1 pswitches, but it can be realized with li pswitches. 
Since l\ < I, this also contradicts our assumption that I is 
minimal. 

If C\ and C2 are connected in parallel, we have 

61 ^ b 2 h b 2 _ a 
qh qh q h q h qn' 

Therefore, 6162 = biq 2 + b 2 q l1 — aq l ~ n . Using a similar 
argument as above, we can conclude that either bi or b 2 is a 
multiple of q. Then either (1) can be realized with less than 
I pswitches or (2) I is not optimal, yielding a contradiction. 
This proves the lemma. ■ 



Based on the lemma above, it is easy to get the following 
theorem. 

Theorem 11 (When q is a prime number larger than 3). For 

a prime number q > 3, there exists an integer a, with < 
a < q n , such that cannot be realized using an sp circuit 
whenever n > 2. 

Proof: The conclusion follows LemmafToland the following 
result in |[T5l : For any q > 3, no pswitch set containing all |, 
with < a < q, can realize all P r (C) = \, with < b < q 2 , 
using at most 2 pswitches. ■ 

V. Probability Approximation 

In this section, we consider a general case where given an 
arbitrary pswitch set, we want to realize a desired probability. 
Clearly, not every desired probability pd can be realized 
without any error using a finite number of pswitches for 
a fixed pswitch set S. So the question is whether we can 
construct a circuit with at most n pswitches such that it can 
approximate the desired probability very well. Namely, the 
difference between the probability of the constructed circuit 
and the desired probability should be as small as possible. 

A. Greedy Algorithm 

Given an arbitrary pswitch set S with \S\ > 2, it is not easy 
to find the optimal circuit (ssp circuit) with n pswitches which 
approximates the desired probability pd- As we discussed in 
the last section, a backward algorithm provides \S\ choices 
for each successive insertion. To find the optimal circuit, we 
may have to search through \S\ n different combinations. As 
|5| or n increases, the number of combinations will increase 
dramatically. In order to reduce the search space, we propose a 
greedy algorithm: In each step, we insert m pswitches, which 
are the "best" locally. Normally, m is a very small constant. 
Since each step has complexity |5| m , the total number of 
possible combinations is reduced to \S\ mJ ^, which is much 
smaller than IS"!" when |5| > 2 and n is large. Now, we 
describe this greedy algorithm briefly. The same notations 
xi, X2, ■ ■ ■ and pi,p 2 , ■ ■ ■ are used, as those described for the 
backward algorithms: Xk indicates the fcth pswitch inserted 
and pk indicates the desired probability of the subcircuit 
constructed by Xk, Xk+i, ■ ■ ■ 

Algorithm 2 (Greedy algorithm with step-length m). 

1) Assume that the desired probability is p\. Set k = 1 and 
start with an empty circuit. 

2) Select the optimal x m = (xi,x 2 , . . . ,x m ) € S m to 
minimize f(x m , S,Pk), which will be specified later, and 
this x m is denoted as x* = (x*, x%, ■ ■ ■ , x* m ). 

3) Insert m pswitches x\,x 2l . . . , x* m one by one into 
the circuit in backward direction. During this process, 
calculate pk+i,Pk+2, ■ ■ ■ ,Pk+m one by one and update 
k as k + m. 

4) Repeat steps 2 and 3 for |_^J times. 

5) Construct a new circuit with n — \_^\m pswitches such 
that its probability is closest to p^ then replace Pk with 
this new circuit. 
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So far, according to the backward algorithm described in 
Section IIV-AI we know how to finish step 3, including how 
to insert m pswitches one by one into a circuit in a backward 
direction, and how to update pk- The only thing unclear in the 
procedure above is the expression of f(x m , S,pk). 

In order to get a good expression for f(x m , S, pk), we study 
how errors propagate in a backward algorithm. Note that in 
a backward algorithm, we insert pswitches x\, X2, ■ ■ ■ , x n one 
by one: if Xk > Pk, then Xk is inserted in series; if Xk < 
Pk, then Xk is inserted in parallel. Now, given a circuit C 
with size n constructed using a backward algorithm, we let 
denote the subcircuit constructed by x kl , x^+i, . . . ,x n 
and call \P(C^ k ') — pk\ as the approximation error of pk, 
denoted by e^. In the following theorem, we will show how 
efcj affects that of ek 2 for k^ < k\ after inserting pswitches 

Xk2 j ■ ■ • j Xki — 1 - 

Lemma 12. In a backward algorithm, let pk denote the 
desired probability of the subcircuit C^ k ' constructed by 
Xk, Xk+i, ■ ■ ■ , x n , and let e% denote the approximation error 
of pk- Then for any k% < k\ < n, we have 
/k 2 -i \ 
efc 2 = J\ r(xi) e kx , 



\ i—ki 



where 



"{Xi) = 



1 - Xi 



if Xi is inserted in series, 
if Xi is inserted in parallel. 



Proof: We only need to prove that for any k less than the 
circuit size, the following result holds: 

efc = r(x k )e k +i. 

When Xk — Pk, we have = ek+i = 0, so the result is 
trivial. 

When Xk > Pk, then Xk is inserted in series. In this case, 
we have 

Pk+lXk = Pk, 

and 

P{C {k+1) )x k = P{C {k) ). 
As a result, the approximation error of pk is 

e k = \P(C^)-p k \ 

= \P(C (k+ V)x k - p k+ ixk\ 
— Xkek+i- 

When Xk < pk, then Xk is inserted in parallel. In this case, 
we have 

Pk+l + X k - Pk+lXk = Pk, 

and 

P{C {k+1) ) +x k - P{C {k+1) )x k = P{C {k) ). 
As a result, the approximation error of pk is 

e k = \P(C^)-p k \ 

= \P(C (k+ V) + x k - P(C^)x k 

-(Pk+l + X k - Pk+lXk)\ 

= (l-x fe )e fc+ i. 



This completes the proof. ■ 

In each step of the greedy algorithm, our goal is to minimize 
efc, the approximation error of pk- According to the lemma 
above, we know that 

Yl r ( X i)\ Sk+rn, 

where the term ek+ m is unknown. But we can minimize 
nSt™ -1 r ( x i) sucn mat e k is as small as possible. 

Based on the above discussion, we express f(x, S,pk) as 

m 

f(x,S,p k ) = Y[r(x l ), (1) 

i=l 

with 

, , f Xi if Xi is inserted in series, 
I 1 — Xi if is inserted in parallel. 

In the rest of this section, based on this expression for 
f{x,S,pk), we show that the greedy algorithm has good 
performance in reducing the approximation error of pd- 

B. Approximation Error when \S\ = 1 

When 5* has only one element, say S = {p}, the greedy 
algorithm above can become really simple. If pk > Pk, then 
we insert one pswitch in parallel; otherwise, we insert it in 
series. Fig. ITT1 demonstrates how to approximate h using four 
pswitches with the same probability i. Initially, p\ 

1 1 1 

so we insert \ in parallel. As a result, P2 = 

we insert the second pswitch in series. The final probability 

of the circuit in Fig. [TT] is |j, which is close to i. 



2^3' 




Fig. 1 1 . This circuit approximates ^ using 4 pswitches of probability i . 

Note that in the greedy algorithm, when p is close to -i, 
the probability of the resulting circuit will quickly converge 
to the desired probability. But when p is close to or 1, the 
convergence speed is slower. In the following theorem, we 
provide an upper bound for the approximation error of the 
desired probability when \S\ = 1. 

Theorem 13 (Approximation error when l^l = 1). Given n 
pswitches, each with probability p, and a desired probability 
Pd, the greedy algorithm (Algorithm^ with m = 1 generates 
an ssp circuit C with approximation error 



\Pd-P(C)\< 



(max{p, 1 — p}) n 
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where equality is achieved when 



Pd = fnip) 



1 - 



(max{p,l— p}) n 



(max{p, 1— p}) r ' 



ifP < i 

ifP > §• 



Proof: In the following proof, we only consider the case 
when p < \- From duality, the result will also hold for p> \. 

We induct on the number of pswitches. For one pswitch, the 
result is trivial: the worst-case desired probability is p+ 
with approximation error Now assume the result of the 
theorem holds for n pswitches, we want to prove that it also 
holds for n + 1 pswitches. 

Let pi = pd be approximated with n + 1 pswitches using 
Algorithmic At the beginning, one pswitch is inserted in series 
if pd < p, or in parallel if pd > p. According to Lemma Q~2] 
we know that the approximation error of p\ is 

ei = r(p)e 2 , 

where r(p) < max{p, 1 — p}, and e 2 is the approximation 
error of p 2 . According to our assumption, we know that 

(max{p, 1 — p}) n 



e 2 < 



So we have 



ei < 



(max{p, 1 - p}) n+1 



Note that equality is achieved if r(p) = max-jjj, 1 — p] and 



(max{p,l-p}) r ' 



In this case, p 2 = f n {p) > \ > P and 



G 2 — > . in Liiu ^nj^, — jn\f) — 2 

the last pswitch is inserted in parallel. As a result, we have 



(1-P) 



n+l 



f n+ l(p) = fn{p)+P - fn{p)p = 1 



as described in the theorem. This completes the proof. ■ 

If we let p = i, the theorem shows that for any desired 
probability pd and any integer n, we can find an ssp circuit 
with n pswitches to approximate pd, such that the approxi- 



mation error is at most 



2 9 " 



This agrees with the result in 
|15|: Given a pswitch set S = {^}, all rational with 
< a < q n , can be realized using at most n pswitches. 



C. Approximation Error when \S\ > 1 

In this subsection, we show that using the greedy algorithm 
(Algorithm [U with small m, such as 1 or 2, we can construct 
a circuit to obtain a good approximation of any desired 
probability. Here, given a pswitch set S = {si, s 2 , . . . , S|s|}, 
we define its maximal interval A as 

A = max|s i+1 - s;|, 

where we let sq — and S|s| + i = 1- In the following 
theorems, we will see that the approximation error of the 
greedy algorithm depends on A, and can decrease rapidly as 
n increases. 

Let us first consider the case m = 1: 

Theorem 14 (Approximation error for m = 1). Assume we 
have the pswitch set S — {si, s 2 , ■ ■ ■ , S|S|} with maximal 
interval A. For any desired probability pd and any integer 



n, Algorithm [2] with m — 1 yields an ssp circuit with at most 
n pswitches, such that the approximation error e satisfies 

< A / (3 + A)A \ rf1-1 

Proof: In the following proof, we only consider the case 
that n is odd. If the result holds for odd n, then the result 
will also hold for even n. In order to simplify the proof, we 
assume that sq = and S|s| + i = 1 also belong to S; i.e., 
there are pswitches with probability or 1. This assumption 
will not affect our conclusion. 

We write n = 2k + 1 and induction on k. When k = 0, the 
result is trivial, since the approximation error of one pswitch 
satisfies e < Assume the result holds for 2k + 1 pswitches. 
We want to show that the result also holds for 2(k + 1) + 1 
pswitches. 

When m = 1 in the greedy algorithm, if we want to 
approximate p\ — pd with 2(k + 1) + 1 pswitches, we should 
insert a pswitch with probability argmin^ f(x, S, pi) in the 
first step, where f(x,S,pi) is defined in ((T). 

Let x up per = min{x € S\x > p{\ and Slower = max{x € 
S\x < pi}. Since e S and 1 6 S, we know that x upper and 

Slower eXISt. 

(1) We first consider the case that 1 — x\ owcr < cc upP er. 
In this case, we insert x\ OWCI in parallel as the first pswitch. 
Therefore, we can get 



P2 



Pi ~ X\ 



1 - Xl 



According to the definition of A, there exists a pswitch 
x E S such that p 2 < x < p 2 + A. Assume in the algorithm, 
we insert pswitch x* as the second one. Since x* is locally 
optimal, we have 

f(x*,S,p 2 )<f(x,S, P2 )< P2 + A. 

Assume the approximation error of ps is e^. According to 
Lemma [12] we know that the approximation error of p\ = pd 
is 



ei < (p2 + A)(l-si 

/ Pi Slower 



(- 



1 Slower 
= ((Pi - Slower 
< A (2 - Slower )e 3 



owcr ) ^3 
+ A)(1 Slower ) 6 3 
) + A(l - £lowcr))e 3 



< 



A(3 + x uppc 



-e 3 



A(3 + A) 

< ^ e 3- 



According to our assumption, 



So 



A (3 + A)A 
~ 2 K 2 ' 



This completes the induction. 
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(2) When 1 — x\ owcr > x upper , we insert ir upper in series as This completes the induction 



the first pswitch. Using a similar argument as above, we can 
also prove that 



ei < 



A (3 + A)A, 



fe+i 



This completes the proof. ■ 

In the next theorem, we show that if we increase to from 
1 to 2, the upper bound of the approximation error can be 
reduced furthermore. 

Theorem 15 (Approximation error for to = 2). Assume we 
have the pswitch set S — {sj, S2, ■ ■ • , S\s\} with maximal 
interval A. For any desired probability pd and any integer 
n, Algorithm\2\with to — 2 yields an ssp circuit with at most 
n pswitches, such that the approximation error e satisfies 

A /(2 + A)A\ 
~ J 



e < 



Proof: As in the proof for m = 1, we only consider the 
case when n is odd, so n = 2k + 1. In the proof, we use the 
same notations as those in the case of m = 1, and assume S 
includes and 1. 

Now we induct on k. When k = 0, the result of the theorem 
is trivial. Assume the result holds for 2k + 1 pswitches; we 
want to prove that it also holds for 2(fc + l) + l pswitches. Let 
^uppcr = min{x G S\x > pi} and £i OW cr = maxjx G S\x < 
pi}, we will consider two different cases as follows. 



(1) If Pi < 



r + A(z U pper+^lo 



we consider 



the following way to insert two pswitches: First insert x\ = 
Slower in parallel, and we get 

Pi ^lowcr 

Pi = -r— • 

-t ^lowcr 

There exists a pswitch X2 G S such that p2 < X2 < P2 + A. 
Then we insert X2 in series as the second pswitch. In this case, 
letting x — (x\, X2), we have 

f(x,S,pi) < (p 2 + A)(l - xi owor ). 

Let x* — (x\,X2) be the two pswitches inserted by the 
algorithm with m = 2, then the approximation error of p\ = 

Pd is 

ei = f(x*,S,p 1 )e 3 < f{x,S,p 1 )e 3 
< {p 2 + A)(l - a;iowcr)e 3 

Pi ^lowcr 



1 



Xlc 



+ A (1 - a; lower )e 3 . 



Since p\ < — 



- — 1 pp — — '-, we have 



ei < 



(x u 



ppcr 



r )(l + A) + A 



-e 3 



A(2 + A) 

< K e 3- 



According to our assumption, we have e 3 < — ( ( 2 +^) A ^ 



so 



') 



ei < - 



A f{2 + A)A \ 



k+l 



(2) If pi > 3: "PP^+ 3; '°wer+A(x uppcr + 3:iowcr 1) ^ we consi( J er 

the following way to insert two pswitches: First insert x\ = 
Supper in series, and we get 

Pi 

P2 = ■ 

•^upper 

There exists a pswitch X2 G S such that P2 — A < X2 < P2- 
Then we insert X2 in parallel as the second pswitch. In this 
case, letting x — (xi,X2), we have 

f{%,S,Pi) < (1 - (P2 ~ A))x upper . 

Let x* = (x'l ,x'2) be the two pswitches inserted by the 
algorithm with m = 2, then the approximation error of p\ = 

Pd is 



ei 



< f(x,S,pi)e 3 

< (1 - (P2 - A))x uppcr e 3 

•^uppcr Pi 



A J X U pp Cr 63- 



Since pi > ^pp^+^^+^-pp^ 1 '- ^ , we have 

- (^upper Slower) (l ~r~ ^) H - ^ 

ei < e 3 

A(2 + A) 
< —J— e 3 . 

Then we have the same result as the first case. ■ 

According to the two theorems above, when we let A — > 
0, the approximation error for m = 1 is upper bounded by 
T (^tO w here k — — 1; and the approximation error for 
to = 2 is upper bounded by A • A fc . It shows that the greedy 
algorithm has good performance in terms of approximation 
error, even when m is very small. Comparing with the case of 
m = 1, if we choose to = 2, the probability of the constructed 
circuit can converge to the desired probability faster as the 
circuit size n increases. 

In the following theorem, we consider the special case S = 
{i, |, . . . , for some integer q. In this case, we obtain a 
new upper bound for the approximation error when using the 
greedy algorithm with to = 2. This bound is slightly tighter 
than the one obtained in Theorem [T5l 



Theorem 16. Suppose S = {|, |, . . . , 

1 For any desired probability pd and any integer 



— } for some integer 

q, with A , . 

n, Algorithm |2] with to = 2 constructs an ssp circuit with at 
most n pswitches such that its approximation error 

e^ACl-A))^- 1 . 

Proof: The proof is similar to the proof of Theorem [15] 
so we simply provide a sketch. Assume that in each step, we 
insert two pswitches in the following way (see Fig. [I2"t : 

(1) If pk G [0, -], we insert a pswitch Xi = - in series, and 
then insert a pswitch X2 = ^ in series or in parallel. In this 
case, 



-) - ) ,S,Pk 
q q 
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Pk + l 



(a) Pk e [0, |] 



-0- q — 1 



(b)p fe e [^,i] 



1/5 1/5 

1-0—0—1 1/5 

1/5 —0—| 

— 1 



2/5 



a + 1 

W Pk 6 [|, f + 1 - f ] tor some (d) p fe 6 [» + i - f , » + i] for 
u = {1, . . . , q — 1}. some u = {0, 1, . . . , q — 2}. 

Fig. 12. Inserting ps witches for different values of p^. 



(2) If pk £ [^-,1], we insert a pswitch X\ = ^— in 
parallel, and then insert a pswitch x% = in series or in 
parallel. In this case, 



(3) Kp fc G [2,2 + |-^-] for some u = {l,. 



, q — 1}, we 

insert a pswitch xi = 2 in parallel, and then insert a pswitch 
^2 = - in series. In this case, 



(4) If Pfc € [f+i-2-, 2+1] for some u = {0, 1, . . . , 9-2}, 

in series, and then insert a 



we insert a pswitch x\ 
pswitch x-i = in parallel. In this case, 



f[[^ q -^),s, Pk 



< 



u + 1 



1 - 



9-1 



< A(l-A). 



Based on the above analysis, we know that for any pf. £ 
(0, 1), we can always find x = (xi, X2) such that 

/((si,X3),S,Pk)) < A(l-A). 

Hence, the result of the theorem can be proved by induction. 
This completes the proof. ■ 

Fig. [13] shows an example for demonstration. Assume 
5 = {gj §}§)§}> an d suppose we want to realize | using 
five pswitches. Using the greedy algorithm with m = 2, we 
can get the circuit in Fig. Qj] whose probability is 0.4278, and 
approximation error is 



0.4278 



7.3 x 10" 



which is very small. 



Fig. 13. The circuit approximates = with 5 pswitches from the pswitch set 
c _ r 1 2 4i 



VI. Conclusion 

In this paper, we have studied the robustness and synthesis 
of stochastic switching circuits. We have shown that ssp cir- 
cuits are robust against small error perturbations, while general 
sp circuits are not. As a result, we focused on constructing 
ssp circuits to synthesize or approximate probabilities. We 
generalized the results in |fl~5l and proved that when q is a 
multiple of 2 or 3, all rational fractions can be realized 
using ssp circuits when the pswitch set S = {|, |, . . . , 
However, this property does not hold when q is a prime 
number greater than 3. For a more general case of an arbitrary 
pswitch set, we proposed a greedy algorithm to construct ssp 
circuits. This method can approximate any desired probability 
with low circuit complexity and small errors. 
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